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1.0  SUMMARY 


The  2012  World  Economic  Forum  declared  data  an  economic  asset  similar  to  gold  or 
currency.  With  ninety  percent  of  the  world’s  data  being  created  in  the  last  two  years,  the 
challenge  to  provide  adequate  security  is  ever  increasing.  Meeting  the  demand  for  top  technical 
cybersecurity  talent  is  one  of  the  continuing  challenges  facing  military  and  civilian  government 
leaders.  On  July  12,  2016,  the  White  House  launched  the  first  ever  Federal  Cybersecurity 
Workforce  Strategy.*  Competitions  continue  to  provide  promising  avenues  for  identification  of 
future  talent,  for  ranking  candidates  on  cyber  skills,  and  for  motivating  candidates  to  become 
fully  committed  to  advancing  their  skills  in  cybersecurity.  The  US  Cyber  Challenge  (USCC)  is 
one  of  the  initiatives  to  address  this  workforce  gap  by  developing  the  next  generation  of  cyber 
experts  through  education  and  hands-on  defense  gaming  strategies.  The  USCC  participated  in 
the  White  House  Cybersecurity  Competition  Workshop  hosted  by  the  Office  of  Science  and 
Technology  Policy  on  July  27,  2016.^ 

By  creating  excitement  and  highlighting  competitors’  successes,  the  USCC  continues  to 
develop  various  avenues  to  communicate  the  following: 

•  There  are  cool  jobs  in  technology; 

•  Organize  highly  selective  summer  camps; 

•  Ensure  the  public  knows  how  impressive  it  is  to  be  selected  to  participate;  and 

•  Analyze  what  works  and  what  does  not  work  in  each  of  these  areas. 

During  the  period  of  performance,  April  2012  through  August  2016,  the  USCC  held  several 
competitions  and  challenges  and  also  developed  the  assessment  framework  to  categorize 
competitions  mapping  them  to  the  National  Initiative  for  Cybersecurity  Education  (NICE)  and 
providing  the  platform  for  social  interaction  with  the  cybersecurity  competition  participants  with 
themselves  and  with  future  employers.  The  impact  of  the  USCC  is  being  measured  against  the 
participation  in  this  platform  called,  CyberCompEx.org  (CCX)  as  well  as  our  future  participation 
with  employers  to  engage  with  the  participants.  Currently,  CCX  has  the  following: 


Table  1:  Current  Statistics  from  CyberCompEx.org 


Community  Members 

2,191 

Total  Topics 

136 

Total  Topic  Replies 

86 

Total  Blog  Posts 

29 

Total  Clips 

325 

In  summary,  the  USCC  continues  our  outreach  and  partnerships  using  the  social  media 
and  other  communications  capabilities  as  well  the  resources  of  the  Center  for  Internet  Security  to 
include  the  Multi-State  ISAC  community  to  increase  the  overall  numbers  of  participations  while 
working  with  partners  such  as  Life  Journey  with  their  5  million  members  to  broaden  the 
information  and  participation  in  CCX  using  the  tools  developed  for  individuals  to  be  able  to 
better  communication  their  skill  sets  to  potential  employers  to  reduce  the  cybersecurity 
workforce  gap. 
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2.0  INTRODUCTION 

In  October  2016,  the  NICE  program  released  the  heat  map  of  the  location  of  the 
cybersecurity  jobs  (http://www.cvberseek.orgl."'^  The  data  states  there  are  348,975  vacancies 
in  the  United  States  alone.  The  Center  for  Strategic  and  International  Studies  (CSIS)  in 
partnership  with  Intel  Security  released,  “Hacking  the  Skills  Shortage:  A  study  of  the 
international  shortage  in  cybersecurity  skills,”  on  July  27,  2016.  This  study  surveyed  eight 
countries,  which  highlights  the  need  for  employers  to  play  in  recruiting,  retaining,  and 
training  their  workforce.^  On  November  1,  2016,  the  USCC  participated  in  the  official  release 
of  the  white  paper  on  the  role  of  cybersecurity  competitions  in  workforce  development  where 
the  USCC  team  participated  in  the  development  through  the  Summer  of  2016.  ^ 

The  Council  on  CyberSecurity  (formerly  the  National  Board  for  Information  Security 
Examiners),  U.S.  Cyber  Challenge  (USCC),  continues  to  develop  partnerships  to  maintain  the 
CCX  platform  for  continuous  engagement  outside  of  the  competitions  and  summer  camps  to 
highlight  career  paths  for  individuals  to  obtain  the  high-level  technical  skills  required  for 
meaningful  employment.  USCC  believes  the  path  must  be  available  through  various  levels  of 
engagement  and  throughout  an  individual’s  career.  Finally,  the  USCC  continues  to  believe 
performance  metrics  should  be  developed  in  order  to  measure  the  progress  being  made  to  reduce 
the  risk  for  the  national  critical  infrastructure  regarding  cyber  threats. 

The  USCC  mission  continues  to  be  the  search  for  10,000  Americans  with  the  skills  to  fill 
the  ranks  for  the  cybersecurity  practitioners,  researchers,  hunters  and  warriors.  Specifically, 

USCC  objectives  are  to: 

1.  Identify:  increase  cybersecurity  knowledge  and  talent  self-awareness  among  high  school 
students,  college  age  students,  current  professionals  and  other  interested  persons  who  are 
looking  to  enter  into  the  cyber  security  professional  ranks  and/or  further  their  existing 
careers; 

2.  Engage:  engage  individuals  in  order  to  increase  the  talent  pool  for  cybersecurity 
professionals,  in  both  the  public  and  private  sectors.  This  objective  includes  engaging 
individuals  across  various  demographics,  including  women  and  minorities;  and 

3.  Challenge:  provide  opportunities  for  skill  development  through  cybersecurity 
competitions  and  pathways  to  provide  increasingly  difficult  challenges  and  competitions 
as  well  as  provide  access  to  educations,  resources,  mentoring,  scholarships,  internships 
and  job  opportunities. 

With  the  development  of  the  CCX  platform,  the  USCC  has  the  ability  to  receive  continuous 
feedback  to  continually  improve  on  these  objectives  and  the  collection  of  the  necessary  data  as 
well  as  establishing  the  data  elements,  conducting  appropriate  collection  activities  and  analysis, 
and  addressing  the  challenges  for  our  on-going  efforts. 
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3.0  METHODS,  ASSUMPTIONS,  AND  PROCEDURES 

3.1  Assumptions 

With  the  initial  work  conducting  the  USCC  used  the  initial  assessment  framework 
developed  under  this  research  proposal,  there  a  number  of  important  assumptions.  These 
assumptions  are  as  follows: 

•  A  more  cybersecurity  aware  populous  will  mitigate  against  the  Nation’s  cyber  risks  and 
improve  the  Nation’s  cybersecurity  posture. 

•  Individuals  with  the  right  skills  and  education  in  cybersecurity,  in  the  appropriate  federal 
and  industry  roles  and  positions,  will  improve  the  cyber  security  posture  of  the 
Nation/National  critical  information  technology  (IT)  infrastructure. 

Given  these  assumptions,  the  initial  assessment  framework  does  not  focus  on  assessing 
or  quantifying  the  Nation’s  cyber  risk  profile  or  cyber  risk  posture  as  this  is  done  at  the  national 
level  and  continues  to  be  expanded  by  reports  provided  by  the  national  intelligence  community. 
The  initial  assessment  framework  was  designed  to  assess  and  improve  the  activities  of  the  USCC 
and  the  cybersecurity  professional  activities  in-line  with  the  objectives  of  identify,  engage  and 
challenge. 

Furthermore,  the  initial  assessment  framework  was  designed  to  support  the  underlying 
USCC’s  “pathway”  construct.  In  this  construct,  the  field  of  cybersecurity  is  viewed  as  a  pathway 
with  multiple  entry  and  exit  points,  as  well  as,  various  paths  to  differing  destinations  for 
education  and  eventual  job  entry  or  if  you  are  already  in  the  job  market,  re-entry  for 
developmental  purposes  to  become  more  cyber-enhanced  and/or  a  cybersecurity  professional. 
However,  since  this  initial  framework  was  developed  and  used  by  the  USCC,  the  analogy  of  a 
“roadway”  with  on  and  off  ramps  works  well.  This  allows  for  the  USCC  to  represent  that  many 
may  be  entering  the  field  at  different  points  of  time  in  their  career,  they  may  be  coming  from 
differing  locations  or  career  backgrounds,  and  ultimately,  they  may  choose  differing  paths  or 
specialties  within  the  cybersecurity  field.  In  contrast,  the  “pipeline”  analogy  continues  to  be 
unfitting  the  USCC  approach  because  it  implies  one  single  point  of  entry  and  highly  structured, 
linear  path  to  a  single  destination.  As  such,  the  information  on  a  participant’s  background, 
experience,  skills,  and  successes  to  help  assist  and  map  a  pathway  through  to  their  choice  and  the 
CCX  platform  was  development  with  this  methodology  in  mind. 

3.2  Methods 

The  initial  assessment  framework  was  geared  towards  measuring  the  success  of  the 
USCC  and  cybersecurity  professional  development  activities  in  achieving  the  USCC’s 
objectives.  As  the  USCC  matured,  we  continued  to  refine  the  methodology  and  the  tools.  We 
developed  two  models  using  experts  across  the  nation  to  address  critical  jobs  at  the  Department 
of  Homeland  Security.  The  Mission  Critical  Role  Project  was  the  scenario-driven  view  of  the 
knowledge,  proficiency  with  tools,  and  abilities  required  for  mission  critical  roles  is  a  framework 
which  we  are  planning  to  use  for  competition  evaluation  design,  and  to  provide  a  way  to 
recognize  competitors  who  score  by  providing  points  toward  their  user  profile  and/or  user 
reputation  index.  The  successful  competitor  would  demonstrate  possession  of  the  knowledge 
required  by  a  specific  competition,  working  proficiency  for  the  tools  needed  to  compete  and  the 
underlying  abilities  exercised.  The  ‘ground  truth’  scenario-based  job  competency  definitions  for 
mission  critical  roles  for  advanced  threat  response  (Security  Monitoring  and  Event  Analyst)  and 
for  operational  security  testing  (System  and  Network  Penetration  Tester)  were  developed  and 
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released  to  the  public. 


3.3  Procedures 

After  deploying  the  initial  assessment  framework,  the  USCC  moved  towards  continuous 
development  and  feedback  cycle  as  depicted  by: 


Figure  1:  Development  Cycle  for  USCC  Activities  and  Partnerships 

4.0  RESULTS  AND  DISCUSSION 


The  Cyber  Camps  provide  crucial  skills  development  and  enable  USCC  to  tap  into  the 
tremendous  talent  across  our  nation  to  identify  those  with  a  passion  for  security  and  a  desire  to 
put  their  skills  to  good  use  in  addressing  our  Nation’s  cyber  security  workforce  challenges.  In 
addition  to  providing  expert  training  for  participants  to  improve  their  skills  and  marketability,  the 
Cyber  Camps  provided  attendees  the  opportunity  to  engage  with  major  technology  companies 
and  government  agencies  at  onsite  job  fairs  for  scholarship,  internship  and  employment 
opportunities  as  well  as  engage  industry  professionals  in  an  ethics  panel. 

The  camps  continue  to  be  "invitation  only"  after  students  initially  completed  the  on-line 
competition.  Cyber  Quest.  Furthermore,  the  camps  were  provided  as  either  day  camps  or  in¬ 
residence  overnight  camps.  For  the  overnight  camps,  there  was  a  minimum  age  requirement  of 
18  years  and  older. 
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Table  2:  Cyber  Quests:  On-line  competition  qualification  round 


Cyber  Quests,  April 


•Registered 

•Complete 


The  Cyber  Quests  chart  illustrates  the  competition  used  for  the  qualification  for  the  camps  only. 
The  USCC  in  conjunction  with  our  partners  from  CounterHackers  developed  multiple 
competitions  and  they  were  conducted  during  various  times  throughout  the  year.  The  qualifying 
competition  for  the  camps  ran  during  the  April  time  period  each  year.  The  two  years  of 
declining  data  are  explained  by  our  focus  on  completion  and  shortened  the  competition  time 
period.  We  concluded  on  the  date  specified  and  did  not  extend  the  last  two  years.  What  we  have 
observed  is  the  participants  wait  until  the  last  minute  to  compete.  They  anticipated  we  would 
extend  the  dates  to  increase  the  participation  rates  and  we  did  not.  We  did  receive  feedback  and 
requests  to  extend  each  year  of  those  two  years. 


Table  3:  USCC  Camps 


Cyber  Camp  Participants 
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The  overall  camp  participation  rate  illustrates  several  findings.  For  example,  in  2011,  the  USCC 
conducted  6  camps  including  one  at  the  high  school  level.  We  developed  and  personally  invited 
all  the  participants.  The  low  number  in  2016  is  the  result  of  only  three  camps  being  conducted. 
The  decline  in  2015  and  the  2016  are  directly  attributed  to  our  partner,  Virginia  Tech.  They  had 
new  staff  who  took  ownership  of  all  aspects  of  the  camps.  With  this  ownership,  the  local  lead 
delegated  the  work  and  the  invitations  did  not  get  sent  out  to  the  potential  camp  participants. 
Also,  Virginia  Tech  did  not  conduct  a  camp  in  2016  due  to  local  staffing  issues.  The  USCC 
continued  to  opt  for  quality  verses  quantity. 

4.1  Social  Media  Statistics  for  August  2016 

Facebook:  August  1, 2016  -  August  31, 2016 

Total  Likes  =  1,887  Total  (growth  of  +0.2%) 

New  Likes  =11 

Weekly  Total  Reach  =  31 

Wall  Posts  (from  fans)  =10  Posts 

Post  Feedback  =  50  Likes,  2  Comments  &  16  Shares 

Gender  Summary  =  78%  Male;  21%  Female;  1%  Unknown 


Twitter: 

July  31 

2,158  Followers 

August  31 

2,196  Followers  (growth  of  +1.8%) 

Linkedin: 

July  31 

1,324  Members 

August  31 

1,324  Members  (growth  of  0%) 

USCyberChallenge.org:  Total  Visitors  =  1,433 

Unique  Visitors  =  1,255 
Page  Views  =  2,426 

CyberCompEx:  August  Registration  =  89 

Twitter  Followers:  146 
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4.2  2016  Capture  the  Flag  (CTF)  Summary 

In  previous  USCC  summer  camps.  Capture  the  Flag  (CTF)  environments  were  built  with 
support  of  various  sponsors  which  while  successful  were  deemed  not  to  be  a  scalable  CTF 
environment.  As  a  result,  USCC  explored  the  question  ‘which  virtual  environment  should  be 
used  for  the  “Capture  the  Flag  (CTF)”  for  the  camps?  And  what  should  be  the  partnership(s)  for 
the  future?’  The  USCC  through  the  years  used  multiple  platforms  primarily  with 
ThreatScapeand  then  this  past  year  with  the  Michigan  Range.  The  CTFs  also  only  tracked 
overall  team  performance.  The  USCC  saw  the  need  for  the  individual’s  contribution  to  the  team 
performance.  The  following  highlights  the  results  of  the  testing  of  the  engine  we  used  during  the 
CTF  competitions  at  the  camps. 


Top  Scores 
B23  points,  247  flags 


Options  »  ^  Connect  USB  Device  »  ^  Send  CtrhAlt'Delete 


capture  the  Fbg 


8.  GaryCohn:  763  points,  241  flags  9.  JustinAndorson:  636  points,  241  Hags 


mPTURE  THE  FLFIG 
C0MPLETEO  CHALLENGES 


J_1NUX  101 


LIBRRRY 


SCHOOL 


TRIVIH 


JayvaunYoung  Forensics  9 
TammyMata  Histoty  31 
AlexReuDen  History  31 
ColeOiLorertzo  Forensks  9 
NoatiAndrews  History  31 


ZSNDR 


\ 


K..  . 


Top  Team  scone 
Boba  Fett:  2660  pts.  247  nags 
Han  solo:  2601  pts,  246  Hags 
Darth  ’^^der:  2457  pts,  245  flags 
Yoda;  2253  pis,  239  flags 
Leia;  2143  pts,  241  flags 
proctors:  2076  pts,  227  flags 
General  Grievous;  2057  pts,  226 
flags 

Oartti  Sidious:  1673  pts,  237  flags 


Tcp  Team  CR  Rettnge 
Boba  Fett:  CR  3 
Han  Solo:  CR  3 
Chewbacca:  GR  2 
Darth  Maul:  CR  2 
proctors:  CR  2 
^a:  CR  2 
Darth  SkJkJUs;  CR  2 
Leia;  OR  2 
Darth  \&der:  CR  2 


Top  independent  Gtate 
I  ParkerGarrison:  1523  points,  246 
flags 

2.  proctorl:  967  points,  227  flags 

3,  AtexReuhen:  965  points.  239 

flags 

4.  LouisTomczaK;  628  points,  247 

flags 

5.  NicholasBnino;  628  points,  247 

flags 


Tap  independent  LR  Rating 
1  ParkerGarrison;  6CR 

2.  proctorl:  4  CR 

3.  AlexReijben:  4  CR 

4.  GaryCohn:  3  CR 

5.  NkholasBruno:  3  CR 

6,  ChaseLucas:  3  CR 

7.  NathanWray;  3  CR 

6.  LouisTonKzak:  3  CR 
9.  MauriceQIack:  2  CR 


Figure  2:  CTF  Display  from  the  Delaware  2016  Camp 
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4.3  2016  Camp  Summary 

The  report  provides  an  overview  of  the  feedback  collected  from  camp  surveys,  a 
summary  of  findings  from  interviews  with  camp  administrators,  teaching  assistants  (TAs)  and 
instructors,  a  summary  of  overarching  observations  derived  from  those  interviews,  as  well  as 
general  recommendations  for  future  camps.  Since  the  inception  of  the  camps,  the  USCC  has 
refined  the  data  collection  mechanisms.  The  full  report  is  included  as  Appendix  B,  US  Cyber 
Challenge  2016  Camp  Analysis.  The  following  is  the  overall  summary  of  the  data  from  the  three 
camps  held  in  Delaware,  Moraine  Valley  and  Southern  Utah. 

Comprehensive  Course  Analysis  Results 


Number  of  Respondents 


Table  4:  2016  Camp  Summary  from  CCX 


Respondent  Summary 

DE 

MV 

suu 

Day  1 

21 

10 

14 

Day  2 

16 

8 

12 

Day  3 

14 

10 

12 

Day  4 

9 

8 

10 

4.4  CyberCompEx.org 

There  are  other  employment  platforms  competing  with  CyberCompEx.org  however,  the 
key  to  our  success  is  our  partnership  with  Monster.com.  The  overall  goal  is  to  have  the 
competitions  be  of  value  to  the  individual  as  they  build  out  their  profiles  and  navigate  their 
career  options  on  the  CCX  and  to  provide  value  to  the  employers  in  order  for  them  to  find 
potential  employees  who  have  demonstrated  the  skills  needed  to  perform  on  the  job.  The  pilot 
for  employers  was  conducted  with  SANS  Institute  and  we  are  continuing  this  effort  going 
forward.  Currently,  we  have  over  12  pending  requests  from  employers  to  participate  on  CCX. 
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CYBERCOMPEX.org 


COMPETITIONS 


CyberCompEx  is  a  virtual 
community  of  talented 
Americans  whose  mission  is  to 
improve  our  country's  ability  to 
anticipate,  avoid,  detect,  and 
defeat  threats  that  endanger 
our  nation. 

Have  you? 

Created  your  account? 

Engaged  with  a  Competition? 
Updated  your  Skills  Profile? 
Viewed  Archived  Competitions? 
Invited  friends/colleagues? 

CALENDAR  EVENTS 


2.  View  Competitions 


Welcome  to 
CyberCompEx! 
Connecting 
America’s 
Best  and 


Cybersecurity 


FEATURED  NEWS 


Post 


Xk  Edit  Custom  Page 

MANAGE  WIDGETS  ON  THIS  CUSTOM  PAGE 

FEATURED  GROUP 

SANS  CyberTalent 

^  CONNECTION 

The  SANS  |  CyberTalent  Connection 
is  a  place  where  employers  and  job 
seekers  can  connect  with  top  Cyber 
Security  Talent  and  jobs. 

»  Job  Seekers  Request  Access/Login 
»  Employers  Request  Access/Login 


JANUARY  28.  2016 


MAIN  CALENDAR 


PIVOT  Project  Contest 
m  All  day 


Cyber  strategy:  ’We  know  what  to  do,  now  we  need 
people  to  do  if  (FCW) 

New  cybersecurity  models  driven  by  tsunami  of  data 


NEW! 


CyberCompex 

Podcasts 


Figure  3:  Screen  shot  of  CCX  including  our  partnership  with  SANS  for  the  Talent  Connection 
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5.0  CONCLUSIONS 

The  success  of  this  effort  and  the  CCX  platform  is  dependent  on  the  continual 
flow  of  data  from  user  participation.  Although  there  is  likely  a  small  population  with  an 
innate  interest  in  continued  participation  over  time,  long-term  analysis  benefits  from  the 
‘pathway’  model  of  incremental  steps  to  include  activities,  education,  achievement,  and 
ultimately  a  benefit,  which  is  the  continual  achievement  and  success  yielding  a  payout  or 
benefit  to  the  participating  individual.  Our  on-going  partnerships  with  the  schools  for  the 
camps,  with  associations  such  as  AFFIRM  for  business  models  to  support  the  camps’ 
scholarships,  Monster.com  for  CCX,  Amazon  Web  Services  for  the  platform  for  the 
USCC  website.  Life  Journey  for  education  outreach,  and  others  will  ensure  the  on-going 
sustainability  of  the  various  models  in  place. 

The  USCC  will  continue  to  use  the  social  media  capabilities  and  other 
communications  capabilities  as  well  the  resources  of  the  Center  for  Internet  Security  to 
include  the  Multi-State  ISAC  community  to  increase  the  overall  numbers  of  participants 
on  CCX  while  outreaching  to  potential  employers. 
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https://www.cybercompex.org/topic/honeynet-project-challenges 

9 

0.1% 

6 

0.6% 

279 

0.1% 

93. 

desktop|ccx|mgs|/channels/mgs1 000/dashboard. aspx 

9 

0.1% 

6 

0.6% 

66 

0.0% 

94. 

https://www.cybercompex.org/topic/waspnet-ctf 

9 

0.1% 

5 

0.5% 

1,100 

0.5% 

95. 

desktop|ccx|hiring|/account/createaccountthankyou.aspx 

9 

0.1% 

4 

0.4% 

1,448 

0.7% 

96. 

https://www.cybercompex.org/topic/cyber-9-12-project 

8 

0.1% 

8 

0.8% 

536 

0.3% 

97. 

https://www.cybercompex.org/event/2016-cyber-security-brainstorm 

8 

0.1% 

8 

0.8% 

133 

0.1% 

98. 

https://www.cybercompex.org/forum/competition-metric-guidelines 

8 

0.1% 

7 

0.7% 

55 

0.0% 

99. 

https://www.cybercompex.org/login/context/GENERAL/redirect/https7o3A7o2F7o2Fwww. 

8 

0.1% 

5 

0.5% 

292 

0.1% 

cybercompex.org7o2Fmember-cp7o2Fupdate-profile 

100. 

https://www.cybercompex.org/forum/ewf-cyber-security-school-challenge 

8 

0.1% 

5 

0.5% 

39 

0.0% 

Total 

7,345 

1,001 

210,870 
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Introduction 

In  2016  the  USCC  hosted  cyber  security  camps  in  multiple  locations  across  the  United  States. 
These  camps  provided  high  school,  college,  and  young  professionals  with  one  week  of 
specialized  cyber  security  training  presented  by  college  faculty  and  cyber  security  experts,  and 
include  a  job  fair  and/  or  capture-the-flag  competition  at  the  various  camps. 

2016  Camp  Schedule 

•  Delaware  State  University  (DE),  July  11th  through  July  15th 
o  Day  1  General  Penetration  Testing 

o  Day  2  Introduction  to  Network  Penetration  Testing 
o  Day  3  Web  App  Ethical  Hacking 
o  Day  4  Metasploit  Kung  Fu  for  Penetration  Testers 
o  Day  5  Capture  the  Flag 

•  Moraine  Valley  Community  College  (MV),  July  20th  through  July  24th 
o  Day  1  Introduction  to  Network  Penetration  Testing 

o  Day  2  Metasploit  Kung  Fu  for  Penetration  Testers 
o  Day  3  Web  App  Penetration  Testing 
o  Day  4  Packet  Crafting  with  Scapy 
o  Day  5  Capture  the  Flag 

•  Southern  Utah  University  (SUU),  July  27th  through  July  31st 
o  Day  1  Introduction  to  Network  Penetration  Testing 

o  Day  2  Web  App  Ethical  Hacking 
o  Day  3  Metasploit  Kung  Fu  for  Penetration  Testers 
o  Day  4  Packet  Crafting  with  Scapy 
o  Day  5  Capture  the  Flag 

The  Cyber  Camps  provide  crucial  skills  development  and  enable  USCC  to  tap  into  the 
tremendous  talent  across  our  nation  to  identify  those  with  a  passion  for  security  and  a  desire  to 
put  their  skills  to  good  use  in  addressing  our  Nation’s  cyber  security  workforce  challenges.  In 
addition  to  providing  expert  training  for  participants  to  improve  their  skills  and  marketability,  the 
Cyber  Camps  provided  attendees  the  opportunity  to  engage  with  major  technology  companies 
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and  government  agencies  at  onsite  job  fairs  for  scholarship,  internship  and  employment 
opportunities  as  well  as  engage  industry  professionals  in  an  ethics  panel. 

This  report  provides  an  overview  of  the  feedback  collected  from  camp  surveys,  a  summary  of 
findings  from  interviews  with  camp  administrators,  TAs  and  instructors,  a  summary  of 
overarching  observations  derived  from  those  interviews,  as  well  as  general  recommendations  for 
future  camps. 
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Survey  Structures 

2016  Survey  Structures 

In  2016,  feedback  on  presenters  and  instructors  was  collected  from  participants  using  the 
CyberCompEx  collaboration  platform.  The  2016  USCC  Evaluation  Survey  (or  simply  Evaluation 
Survey),  was  given  to  participants  to  evaluate  the  setup,  execution,  and  expectations  for  the 
event. 


2016  USCC  Evaluation  Survey  Structure 
1.  Course  Evaluation 

1.1.  On  a  scale  of  1-10,  what  is  your  overall  evaluation  of  this  course? 

1.1.1.  l,Bad 

1.1.2.  2, 

1.1.3.  3,  Poor 

1.1.4.  4, 

1.1.5.  5,  Marginal 

1.1.6.  6, 

1.1.7.  7,  Good 

1.1.8.  8, 

1.1.9.  9,  Great 

1.1.10.  10,  Excellent 

1.2.  On  a  scale  of  1-10,  what  is  your  overall  evaluation  of  the  instructor's  teaching  skill? 


1.2.1. 

l,Bad 

1.2.2. 

2, 

1.2.3. 

3,  Poor 

1.2.4. 

4, 

1.2.5. 

5,  Marginal 

1.2.6. 

6, 

1.2.7. 

7,  Good 

1.2.8. 

8, 

1.2.9. 

9,  Great 
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1.2.10.  10,  Excellent 


1.3.  On  a  scale  of  1-10,  what  is  your  overall  evaluation  of  the  value  of  the  course? 

1.3.1.  l,Bad 

1.3.2.  2, 

1.3.3.  3,  Poor 

1.3.4.  4, 

1.3.5.  5,  Marginal 

1.3.6.  6, 

1.3.7.  7,  Good 

1.3.8.  8, 

1.3.9.  9,  Great 

1.3.10.  10,  Excellent 

1.4.  If  you  have  any  additional  comments  or  feedback,  please  provide  it  here. 

1.4.1.  Free  form  data  entry. 


2.  Capture  the  Flag  Evaluation 

2.1.  Were  the  TAs  helpful  in  explaining  the  set  up  for  this  event? 

2.1.1.  Yes 

2.1.2.  No 

2.2.  On  a  scale  to  1-5,  how  would  you  rate  the  quality  of  the  teehnieal  doeumentation,  if  any 
provided  by  the  TAs? 

2.2.1.  1,  Inadequate 

2.2.2.  2,  Poor 

2.2.3.  3,  Aeeeptable 

2.2.4.  4,  Good 

2.2.5.  5,  Exeellent 

2.3.  Did  the  competition  appear  to  provide  the  technical  challenge  for  the  class? 

2.3.1.  Yes 

2.3.2.  No 

2.4.  Do  you  fell  the  technical  set  up  was  easy  to  aehieve? 

2.4.1.  Yes 
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2.4.2.  No 

2.5.  Did  the  technical  performance  meet  your  expectations? 

2.5.1.  Yes 

2.5.2.  No 

2.6.  On  a  scale  of  1-10,  what  is  overall  evaluation  of  this  event? 


2.6.1. 

l,Bad 

2.6.2. 

2 

2.6.3. 

3,  Poor 

2.6.4. 

4 

2.6.5. 

5,  Marginal 

2.6.6. 

6 

2.6.7. 

7,  Good 

2.6.8. 

8 

2.6.9. 

9,  Great 

2.6.10.  10,  Excellent 

2.7.  What  are  the  strengths  of  this  competition? 

2.7.1.  Free  form  data  entry. 

2.8.  What  are  the  areas  for  improvements  for  this  competition? 

2.8.1.  Free  form  data  entry. 

2.9.  Would  you  recommend  this  competition  to  your  peers? 

2.9.1.  Yes 

2.9.2.  No 

2.10.  If  you  have  any  additional  comments  or  feedback,  please  provide  it  here 

2.10.1.  Free  form  data  entry. 
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Comprehensive  Course  Analysis  Results 


Number  of  Respondents 


Respondent  Summary 

DE 

MV 

suu 

Day  1 

21 

10 

14 

Day  2 

16 

8 

12 

Day  3 

14 

10 

12 

Day  4 

9 

8 

10 
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On  a  scale  of  1-10,  what  is  your  overall  evaluation  of  this  course? 
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On  a  scale  of  1-10,  what  is  your  overall  evaluation  of  the  instructor's  teaching 
skill? 
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On  a  scale  of  1-10,  what  is  your  overall  evaluation  of  the  value  of  the  course 
content? 
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Comprehensive  CTF  Analysis  Results 


Number  of  Respondents 


Respondent  Summary 

DE 

MV 

suu 

Total 

11 

5 

10 

Were  the  TAs  helpful  in  explaining  the  set  up  for  this  event? 
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On  a  scale  to  1-5,  how  would  you  rate  the  quality  of  the  technical  documentation, 
if  any  provided  by  the  TAs? 


Did  the  competition  appear  to  provide  the  technical  challenge  for  the  class? 
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Do  you  fell  the  technical  set  up  was  easy  to  achieve? 


Did  the  technical  performance  meet  your  expectations? 
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On  a  scale  of  1-10,  what  is  overall  evaluation  of  this  event? 


Would  you  recommend  this  competition  to  your  peers? 
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List  of  Acronyms 


AFFIRM 

CCX 

Multi-State  ISAC 


CSIS 

CTF 


Multi-State  ISAC 


TAs 


USCC 


The  Association  for  Federal  Information  Resources  Management 
CyberCompEx.org 

Multi-State  Information  Sharing  and  Analysis  Center 
Center  for  Strategic  and  International  Studies 
Capture  the  Flag 

Multi-State  Information  Sharing  and  Analysis  Center 

Teaching  Assistants 

United  States  Cyber  Challenge 
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